The Di culty of Reduced Error Pruning ofLeveled Branching

Induction of decision trees is one of the most successful approaches to supervised machine learning. Branching programs are a generalization of decision trees and, by the boosting analysis, exponentially more eeciently learnable than decision trees. In experiments this advantage has not been seen to materialize. Decision trees are easy to simplify using pruning. For branching programs no such algorithms are known. We prove that reduced error pruning of branching programs is infeasible. Finding the optimal pruning of a branching program with respect to a set of pruning examples that is separate from the set of training examples is NP-complete. Therefore, we are forced to consider approximate solutions to this problem. We also prove that nding an approximate solution of arbitrary accuracy is computationally intractable. In particular, the reduced error pruning of branching programs is APX-hard.